Finding Global Optimum for Truth Discovery: Entropy Based Geometric Variance

Truth Discovery is an important problem arising in data analytics related fields such as data mining, database, and big data. It concerns about finding the most trustworthy information from a dataset acquired from a number of unreliable sources. Due to its importance, the problem has been extensively studied in recent years and a number techniques have already been proposed. However, all of them are of heuristic nature and do not have any quality guarantee. In this paper, we formulate the problem as a high dimensional geometric optimization problem, called Entropy based Geometric Variance. Relying on a number of novel geometric techniques (such as LogPartition and Modified Simplex Lemma), we further discover new insights to this problem. We show, for the first time, that the truth discovery problem can be solved with guaranteed quality of solution. Particularly, we show that it is possible to achieve a (1 + )-approximation within nearly linear time under some reasonable assumptions. We expect that our algorithm will be useful for other data related applications. 1998 ACM Subject Classification F.2.2 Nonnumerical Algorithms and Problems Geometrical problems and computations